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Abstract 



This paper is an analysis of how teachers evaluate and keep track of students' literacy 
development, particularly as it relates to the reading of literature. We interviewed teachers 
from a range of school contexts. We asked them, among other things, how they kept track of 
students* literacy development, and to describe particular children's development with respect to 
literacy and literature. We found that teachers' evaluative descriptions of students' development 
in literacy differed in: a) sheer quantity of information; and b) the extent to which their de- 
scriptions featured students' knowledge of, and preference in, literature, and the extent of 
"personal" as opposed to "objective" knowledge. These differences were influenced by the 
extent of teachers' knowledge about literature and the conditions under which they taught. In 
general, elementary school teachers knew more about students than did high school teachers, 
because of sheer numbers of student charges and time spent with them. However, the descrip- 
tions of students' development given by those elementary teachers who were strictly required to 
use a basal reading program and under heavy threat of accountability testing, did not reflect the 
extent of the teacher's knowledge of literature. The language of teachers' descriptions also 
differed depending on whether they were describing a more or less able student. 



Teachers' methods of keeping track of children's literacy development reflect their goals 
and values in the teaching of literacy and literature, but also reflect the constraints under which 
they work. Their evaluations of their own effectiveness in the teaching of literacy and litera- 
ture are also closely allied to these values, except in a context which does not support them. 
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Teachers 9 Evaluations of Teaching and Learning 
in Literacy and Literature 

Peter H. Johnston 
Paula Weiss 
State University of New York 
University at Albany 

Peter Afflerbach 
Emory University 

Although current debates about evaluation revolve around standardized testing, standard- 
ized tests are only a part of the evaluation of literacy development in the classroom. Evaluation 
ranges from the moment-to-moment interpretations and responses made by teachers over the 
course of the day to responses made to assignments, to the use of various formal and informal 
tests, and to the writing and interpretation of report cards. Research on the interactions be- 
tween teachers and students suggests that the instructional interaction is driven, in part, by the 
teacher's conceptions of (evaluations of) the student, of the domain of study, and of how chil- 
dren learn (Allington, 1983; Hiebert, 1983; Page, 1987; Peterson & Comeaux, 1987; Weiss, 
Johnston, & Colfer, in press). 

Recent literature on teachers and teaching also portrays teachers as decision-makers 
(Shavelson & Stern, 1981, 1983), as researchers (Goswami & Stillman, 1987), as reflective practi- 
tioners (Schon, 1983), and as evaluation experts (Johnston, 1987). These views of the teacher all 
depend on the extent and organization of teachers 9 knowledge of children's development in 
literacy, their knowledge of literature, and their knowledge of themselves as teachers of literacy 
and literature. At the same time, there is an increased awareness that teachers do not all hold 
the same set of beliefs about these issues (Bussis, Chittenden, & Amarel, 1976; Nespor, 1987; 
Roehler, Duffy, Hermann, Conley, & Johnson, 1988), hence what they observe and how they 
interpret what they observe differs from teacher to teacher in critical and value-laden ways. 
Their evaluations of children's development have consequences for literacy and literature curri- 
cula as they are played out in the classroom and in the ways they talk about children's develop- 
ment with each other, with parents, and with students. 

O the other hand, the conditions under which a teacher works are also likely to have an 
impact on what knowledge is seen as important, or even accessible, and may constrain the 
metaphors of teachers' views of themselves and their students. For example, if a teacher uses 
only a basal reading program in the classroom and no other literature is available, this is likely 
to influence the literate activity available for him to observe. Students responding to stories 
through workbooks provide different behaviors to observe than do students responding in dia- 
logue. At the same time, if teachers are only supplied with a basal reader, are required to use it 
in detail, and are not supplied with children's literature for the classroom, there is a good 
chance that they will not develop a wide knowledge of literature, unless they have ample time 
at home for hobbies, and they happen to choose literature, particularly children's literature, as 
their hobby. If teachers are required to have students work in a basal reader, in which progress 
is presented as a linear progression through a series of skills and increasingly difficult text, then 
that linear sequence, stripped of alternatives, is likely to acquire high salience, especially if it is 
reinforced by a testing system. 



Indeed, several writers have explored the consequences of conditions such as these in 
which the need for teachers to make such instructional decisions is reduced, and in which 
teachers are simultaneously deprived of the confidence and expertise to make such decisions. 
For example, Shannon (1988) has described the technical control of teachers through commercial 
reading materials. Although these materials may contain children's literature, they also contain 
instructions specifying students' activity, and tests and worksheets for children to complete and 
teachers to evaluate. Shannon notes that: 

Accepting basal goals and means for instruction, teachers confine their activities to 
managing students through materials by applying guidebook directives for smooth running 
lessons designed to ensure that students pass the program's tests, (p. 93) 

This has clear implications for the ways in which teachers are likely to view students. 
Shannon also points out that these programs restrict teachers 9 decision-making without even 
appearing to do so as they become an increasingly obvious and natural part of the job of teach- 
ing reading. This view also holds potential implications for the ways in which teachers evaluate 
their own teaching practice. 

Apple (1982) describes an additional type of control which can compound the more 
subtle technical control. Bureaucratic control operates through institutional rules, roles and 
regulations, which can both limit a teachers' decisions, and simultaneously provide threat of 
sanctions if they do not teach in the prescribed manner. Apple argues that these types of 
control can lead to an intensification of the work of teaching so that teachers become so in- 
volved in simply keeping up, that they can only attend to the H essentials, M which are those as- 
pects most clearly demanded by the system. 

In other words, we have two different sets of research, one describing teachers as if they 
were autonomous, knowledgeable decision-makers, and one which specifies conditions under 
which this autonomy and authority is undermined. The present study set out to examine the 
intersection of these theoretical issues. The focus of the study is teachers' evaluations of their 
teaching and of the students' learning of literacy and the reading of literature in more or less 
controlled professional contexts. We were particularly interested in the knowledge teachers 
bring to bear in evaluating their teaching and their students' learning. We view teachers 1 class- 
room evaluation much as we view reading. Just as teachers might read a book, they read their 
students' and their own performance, raising all of the interpretive issues that that implies. Just 
as the situation in which we read a book has an impact on our reading, so it seems likely that 
the context in which teachers work will influence their reading of their students learning and 
their own teaching practice. 



Method 

Because the focus of this study was on teachers' personal interpretations, semistructur*d 
interviews were considered the most appropriate type of data. Spradley (1979) has described in 
detail the structure and method of the ethnographic interview, and we followed as closely as 
possible his approach. We were, however, generally limited to a single interview rather than 
multiple interviews, ! n order to obtain as many interviewees as possible, and to make the study 



acceptable to the authorities involved. 



Sample Characteristics 

The teachers in the study were volunteers from five school districts. Four of these dis- 
tricts were relatively small (four to six thousand students) suburban districts. In one of these 
districts, the teachers were actively encouraged by administrators not to use the basal reading 
program which was available to them. They were provided with professional libraries, financial 
support for the development of classroom libraries, and encouraged to develop their own re- 
cord-keeping techniques. In this district, at least three teachers participated at each of the 
grade levels 1, 2, 4, 5, 8, 9, and 11 (some taught at more than one level), including 22 teachers. 

By contrast, in three other suburban districts, administrators did not enforce the use of 
the basal reader, but did not discourage it either. Thus, some teachers remained fully commit- 
ted to the use of the basal program, and others were in varying stages of displacing it from 
their practice. Sixteen teachers from these three districts participated in the study. These teach- 
ers taught at grades 1(4), 2(3), 2-3(1) 3(1), 4(7). 

A fourth group of thirteen teachers came from a large urban district. These teachers 
taught grades 1(3), 3(3), 4(1); two taught middle school and four taught high school. In this 
district, the use of the basal reading program was strictly enforced. Teachers were required to 
turn in end-of-unit tests to the district office to be scored. Their students could be "spot-test- 
ed" by administrators, with 24 hours notice, on topics they should have covered in the basal at a 
given point in the year. Although all districts in the study had external testing programs with 
varying degrees of visibility, this district had the most intensive and highest profile testing 
program by a substantial margin. 

The teachers in the study thus came from districts which represented a range of program 
focus and technological and bureaucratic control. The extremes were essentially a focus on 
students' independent reading of children's literature, with minimal control of teachers' instruc- 
tion, through a mixture of basal reading program and literature but largely technological con- 
trol, through an enforced emphasis on the basal reader in a situation of extreme technological 
and bureaucratic control. For ease of discussion, we will refer to the extremes of this continu- 
um as the "literature" programs and the "controlled" programs. All teachers in the study were 
volunteers. 



Interview Procedure 

Teachers were interviewed either after school or during preparation periods at the teach- 
ers' convenience. Teachers were either interviewed in the school building (mosi in the class- 
rooms) or in a teacher center. Most teachers were interviewed in a single session, though some 
required two sessions. Interviews ranged from three quarters of an hour to nearly two hours. 
The interview was formal and semi-structured; the relevant questions are shown in Appendix I. 
Interviews often did not follow the same sequence of questions since every effort was made to 
encourage the teachers to explore the issues from their own perspectives and to talk about issues 
in their own terms. This allowed us to study what teachers chose to foreground, and the terms 
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they used to structure their comments. The major relevant topics covered in the interviews 
were: 

• Descriptions of students' literacy development 

• Teachers' knowledge of literature 

• Classroom literacy teaching practice and priorities 

• Classroom evaluation practice and priorities 



Analysis 

Audio tapes of the interviews were transcribed and analyzed. Analyses have attempted to 
stay true to the words, thoughts, and meanings of those interviewed by being sensitive to the 
type of language used by teachers to talk about what they know and to the number of examples 
and repetitions of comments. Interview analyses began with a full domain analysis on five 
initial transcripts. Four researchers spent a full day reading and classifying issues raised by 
teachers in the transcripts and issues raised by the language they used. These researchers then, 
as a group, sought ways to cluster the issues into domains. Six domains eme;ged from this ini- 
tial analysis. The domains were converted to questions which could be asked of all of the tran- 
scripts. For example, the domain cover term: "kinds of qualifications this teacher places on the 
professional knowledge he/she has" became "What qualifications does this teacher put on his/her 
(professional) knowledge related to the evaluation of literary development?" The six resulting 
questions which were "asked" of the data were: 

1. How do teachers know how they are doing in the teaching of literacy and literature? 

2. In what ways does literature come up when teachers talk about students' literacy 
development? 

3. How do teachers get knowledge of their students' development with respect to litera- 
cy and the reading of literature? 

4. What do teachers know about individual students' preferences in literature? 

5. What impact does a teacher's literary preferences and knowledge about literature have 
on the evaluation of children's development in the area? 

6. What qualifications does this teacher put on his/her (professional) knowledge related 
to the evaluation of literacy development? 

All transcripts were read completely at least twice and transcript segments relevant to 
these questions were coded. Segments were tabulated by grade (elementary, middle, high school) 
within district, by nature of response, and by teacher (coded within level). An answer was then 
sought to each of the questions by studying the coded transcript segments and the pattern of the 
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tabulated responses. Answers to the questions, along with multiple readings of coded sections of 
all of the transcripts, yielded some redundancies across questions and some additional patterns, 
particularly concerning the differing contexts in which teachers worked. Thus the structure 
which emerged from the entire corpus was slightly different from the initial analysis. 



Results and Discussion 

Results will be presented as summary statements derived from the data followed by 
discussion and illustrative examples of the data which produced the statement. For the sake of 
readability, most examples are placed in appendixes, separated from the text. 

The first major summary statement was based on teachers 1 descriptions of specific 
students' literacy development. We concluded that: Teachers' evaluative descriptions of students' 
development in literacy differ in: 

a) Sheer quantity of information, 

b) The ways the descriptions were framed, including the extent to which their descrip- 
tions featured students' knowledge of, and preference in, literature, and the extent of 
"personal" as opposed to "objective" knowledge presented. 

All of our interviews were undertaken after at least two months of classes had been 
completed. Although we might reasonably expect teacher's descriptions of students' develop- 
ment to differ at different points during the year, we might expect that when teachers choose a 
student whose development they know best, they will have had time to establish a reasonable 
base of knowledge about that student. However, even after seven or eight months of school we 
found substantial differences between teachers' descriptions. Some teachers, even with prompt- 
ing, produced only three or four lines of transcribed description. Others, without prompting, 
produced two or three pages of single-spaced transcript on a single child, and repeated the 
performance on others. A contrast illustrating this range is shown in Appendix A. 

Not only did the descriptions differ in quantity, but they differed in nature. Some 
teachers framed their descriptions in terms of the literature which the student selected, how it 
changed, how he or she responded to it, and how aspects of it appeared in his or her writing. 
Examples of these descriptions are provided in Appendix B (but examples of this framing can 
also be seen in Appendixes A and C). None of these descriptions came from a teacher in a 
highly controlled context. 

Other teachers did not mention literature at all. Some framed their comments in terms of 
the "level" at which the child performed, or his or her non-academic behavior. Contrasting 
examples representing these differences are shown in Appendix C. 

Some of these descriptions differed in terms of the extent to which they used more 
personal, "subjective" language as opposed to more "objective" or "clinical" language. This dif- 
ference would often show up at the very beginning of a teachers' description when he or she 
might essentially introduce us personally to the student and his or her personality before going 
on to describe the literate activity we had asked for, as opposed to beginning with a statement 
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of reading level. Another way in which it showed up was when teachers, when asked to de- 
scribe a particular child, in spite of prompting, kept talking about the group, or about what he 
or she as a teacher was doing for the child. 

The second major summary statement concerns the conditions under which these differ- 
ences in teachers' descriptions occurred. 

Teachers' evaluative descriptions of students' development in literacy were influenced by the 
extent of teachers' knowledge about literature and by the context of their teaching practice. In 
general: 

a) Teachers' knowledge about literature tended to be reflected in their descriptions of 
children's literacy development. 

b) However, the extent to which they did so was inversely related to the extent of their 
use of the basal reading program. 

c) Elementary school teachers tended to know more about their individual students than 
did high school teachers. 

d) In contexts of moderate technical control descriptions of more able students tended to 
feature literature more than did descriptions of less able students. 



We estimated teachers' knowledge about literature by asking them what books they had 
read recently, and asking them to classify three of those books in as many ways as they could. 
We asked "How is one of these books different from the other two?" and "Is there any other 
way to classify those books?" When teachers could not remember the titles of any books read, 
we assumed a limited knowledge of literature. At the other extreme, when they were able to 
give titles, authors, and to classify the books in more than one way, we assumed a considerable 
knowledge of literature. In general, when teachers knew more about literature they tended to 
describe their students in terms of their selection and reading of literature. An example of this 
contrast can be seen in Appendix D. 

Descriptions of students' literacy development provided by elementary teachers were 
almost all of greater length than those provided by teachers in the higher grades. This should 
not be surprising, as the upper grade teachers were trying to keep track of perhaps 150 students' 
development while seeing them for 45 minutes a day. No teacher in such a situation gave us 
descriptions longer than 25 lines of transcript. This contextual restriction also limited the extent 
to which their knowledge of literature was reflected in their descriptions of the students. In 
general, teachers in the upper grades gave evidence of substantial knowledge of literature, but 
their descriptions of their students' development did not reflect that knowledge in the way that 
some of the elementary school teachers' descriptions reflected their knowledge. 

On the other hand, some of the briefest descriptions came from elementary school teach- 
ers in the highly controlled context. Recall that those teachers taught in a situation of intense 
accountability in which they were required to follow a basal reader in detail, and to have the 
end-of-unit tests sent to the head office to be scored. They also faced an extensive external 
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testing program. Time was seen to be at a premium. These teachers did not have the opportu- 
nity to observe their students reading literature and to talk with them individually about it. 
Indeed, they often found that they had not time even to read literature to their students. When 
asked "Have any books been read lattly in class?" one teacher responded: 

No. None. I desire to do that but there is absolutely no time with the pacing. It makes it 
impossible. There is no time . . . 

Unlike in the less controlled districts, children*.* literature was not supplied to these classrooms, 
nor were funds provided for teachers to purchase literature. 

When teachers were constrainea to follow a basal reader and evaluation was test-driven, 
even though they might know and love literature, this knowledge did not show up in their 
descriptions of children's literacy development. Where literature dominated the classroom, teach- 
ers generally viewed students 9 development through the books they were choosing to read and 
discuss, and through the reflections of those books in their writing. Where tests and basals 
ruled the classroom, descriptions of development were framed by tests. In this latter situation, 
descriptions of the students* development were relatively scant and tended to be impersonal, 
reflecting a lack of detailed knowledge of the students and a less personal, and involving rela- 
tionship. In the highly controlled context, one teacher who had extensive knowledge of litera- 
ture and who managed to read literature to the children every day, described a child whose 
literacy development she knew well without any reference to literature at all. 

Another pattern which emerged among those teachers who were knowledgeable about 
literature was that although they may have spoken of more able students in terms of the litera- 
ture being read, literature tended not to occur in their descriptions of their less able students. 
For example, an eighth-grade teacher in a literature oriented context seemed to be very inter- 
ested in, and kept close track of, literature that his students were reading, and also was able to 
talk extensively about several books that he read recently, unrelated to his teaching. Not sur- 
prisingly, when he was asked in his interview to describe a student's lueracy development, his 
description of an "upper level student" was framed by his knowledge of the students' progres- 
sion through various types of books, and the discussions the teacher had had with this student 
about those books. However, when describing a "lower level student" this same teacher did not 
mention literature at all. These contrasting excerpts from the interview appear in Appendix L. 

A possible cause for this was described by this teacher: 

I know a lot of the higher level students. Because they talk a lot to me about literature, 
and what they're reading, what they like to read. . . [whereas with the lower level stu- 
dents] ... I don't know their literacy level that well which is the problem. . . probably 
because he only talks about literature when he has to talk about literature, and he 
doesn't really care much about literature. He doesn't care much about school or writing 
either. It's very hard for me to get something out of him to try to find out where he is, 
or where he's at and where he can move to... 

Although this was not an isolated case, the extent of the pattern was limited because 
only some teachers who had the opportunity for the children to read literature, and knew 
something about literature, selected a more and a less able student to describe. 
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The third major summary statement concerns teacher's evaluation techniques. Teachers 
differed substantially in the ways they used to evaluate their students' development. Although 
virtually all of the teachers mentioned classroom observations of their students, there were 
considerable differences between teachers in the ways they used to keep track of students' 
development. These differences were most clear between the most strongly literature-based 
context and the most strongly controlled context. 

First, there was a difference in emphasis on direct teacher observations as opposed to 
more formal tests. In the district which actively encouraged literature-based instruction, virtual- 
ly all of the teachers' responses to the question "How do you know how students are doing in 
their literacy developm. nt?" involved teacher observation of student behavior and student talk 
about books. This emphasis* was particularly strong in the elementary schools. Teachers in this 
district described the following techniques: 

• running records - a written record made of student oral reading behavior while the 
student reads a book. 

• regular written narratives of students' development. 

• ring-binder with tabbed sections to record comments on each student based on 
individual interviews and classroom discussions. 

• records of books read by each student. 

• dialogue journals relating to students' reading of literature. 

These records were based on classroom observations, individual conferences with stu- 
dents, students' writing in journals, and tests, particularly informal, individualized ones. Seven 
teachers from that district mentioned holding frequent individual conferences with students, and 
eight said that they used journals, sometimes on a daily basis. 

Tests were mentioned in this district by sixteen teachers as playing at least a supporting 
role in following student literacy development. However "tests", as they described them, often 
meant teacher-made or individually administered informal tests. The pattern of responses was 
essentially the same in both upper and lower grades. In the uppe/ grades there was the addi- 
tional category of assignments, and the tests were more likely to be class tests than individual 
ones. 

At the other end of the continuum, in the highly controlled setting, the primary category 
of ways of keeping track of students' development was also observations of students' behaviors 
and comments. However, the descriptions of these informal means provided by these teachers 
were substantially less specific, and the recording methods less formalized. Statements of the 
value of observing and listening to students were often not supported by examples, and the 
examples that vere giver, always were less extensive, and unlike those from the less controlled 
contexts, never included reference to children's literature. Furthermore, "serious" testing was 
emphasized in every grade in the highly controlled situation. Although all these teachers men- 
tioned tests as a large part of their programs, some were skeptical about their usefulness. 
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In the highly controlled situation, no teacher mentioned conferences or journals but they 
did mention workbooks and assignments. In the literature district only one teacher mentioned 
assignments or workbooks. The language used to talk about these observations was also differ- 
ent. For example, teachers in the more basal-oriented programs (or those most recently using 
them), used the term "mastery" frequently whereas it was not used in the literature-based dis- 
trict. The term "skills" occurred with greater frequency in the more controlled contexts also. 
Descriptions given by those who only recently moved out of a basal reader tended also to focus 
on "skills" and not to mention choice of literature, even though children were choosing books in 
the classroom. Similarly, although "class discussion" was referred to by a wide range of teach- 
ers, the teachers in the more basal-oriented programs used it in the context of "questions and 
answers" or "about literal information" and other more teacher-directed terms. 

In the highly controlled district, the second most heavily weighted source of information 
was formal testing, and it was second to observation by a narrow margin. Indeed, although 
observations constituted the primary category, observations were most often described essentially 
in testing terms. 

The fourth summary statement follows directly from these observations. Teachers' evalua- 
tions of children's literacy development were related to the context in which they taught and the 
goals they expressed. The indicators of literacy development which the literature-based elemen- 
tary and middle school teachers mentioned in relationship to literature included the following: a 
wide knowledge of authors and illustrators, types of language used in various books, choice of 
literary genre, ability to classify books into various categories, number of books read, affective 
responses to book% awareness of different interpretations/levels of meaning, differentiation of 
purposes for reading, choosing books of appropriate difficulty, and knowledge of story struc- 
ture. 

Elementary and middle school teachers from the most controlled district also spoke of 
literature-related indicators of literacy development. Some spoke of enjoyment and enthusiasm 
for books. However, they were much more likely to mention more objective factors such as 
end-of-unit tests, specific skill attainment, standardized tests, number of books read, book 
reports, or spelling bee performance. This was also true of the teachers who were not required 
to use the basal program, but who did so and were in a reasonably high pressure accountability 
situation. 

When teachers were asked directly, "What do you think your students should know about 
literature before they leave your class?" their descriptions were generally quite compatible with 
their comments about children's literacy development and their choices of indicators of devel- 
opment. The major category of goal for the teachers in the less controlled contexts was that the 
students love literature. This was particularly marked in the elementary school. In contrast, 
only half of the elementary teachers in the highly controlled context noted this as a goal. The 
second goal for the literature-based teachers was that students be exposed to and able to talk 
about a wide variety of literature and authors' styles. A third goal for these teachers was that 
children's knowledge of literature should show up in their writing and a fourth goal had to do 
with becoming more selective about what is read - developing taste. The elementary school 
teachers in the literature-based setting were already talking about knowledge of character devel- 
opment, plot structure and style as important goals. These became more important in the 
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middle school and high school. 



The teachers in the highly controlled setting, by and large, felt that students should be 
more able to answer questions about what they read. They also expressed a general feeling that 
students were not reading as much literature as they perhaps should be. This turned out to be 
an important determinant of what was possible in evaluating literacy development. If children 
were not reading literature it was not possible to observe and discuss their reading of literature. 
On the other hand, some teachers knew little about literature. Their goals, descriptions and 
methods of evaluation reflected this. These teachers* descriptions were also more likely to 
emphasize competitive attainments and test-like language than were those of teachers with a 
greater emphasis on literature and in a less controlled setting, who tended to emphasize enthusi- 
asm, cooperation and literature. An extended example reflecting a coherent relationship be- 
tween goals and evaluation is shown in Appendix F. 

The influence of ext< rnal testing (and externally scored classroom tests) could be found 
extensively from the first grade on in the highly controlled district. In general, goals focused 
on skills and the tests, as did teachers' methods of evaluating progress. Two responses which 
illustrate this perspective and its context are shown in Appendix G. 

This was not, however, universal among these teachers. For example, one teacher in this 
context provided a more personal and detailed set of literacy goals for her students: 

Well, I hope that they will be able to pick out a mood, like exacdy how people are 
feeling when certain things were happening. They would be able to personalize, or put 
themselves in the character's place, a little critical thinking, imaginary things. Like a lot 
of the stories at the elementary level are fiction, so it takes a lot of imagination - the 
setting, the time, the plot. But just that fact that they enjoy reading, just to sit down 
and pick up anything and read it and enjoy it. 

However, when discussing indicators of her students* development, this set of goals was 
not reflected to the extent that goals were reflected in the descriptions provided by the teachers 
in the less controlled, more literature-based contexts. When asked how she knew they had at- 
tained these goals, this teacher said: 

By how they respond, and I can see it in their eyes. Their faces give everything away, 
y'know? So if they have their hands up or not, you can look at them and know, cause 
their eyes tell. 

When asked how she knows how her students are doing in class in reading and writing, 
she commented: 

I depend on tests as far as letting me know what they have grasped. . . I give a test 
every Friday in every subject. . OK?. . . and that would be for me so I can get an idea 
of where to go the next week as far as lesson plans. And the end of the month tests, 
those are more of a composite. That's the one I would count in my grade books. 
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Those teachers in the highly controlled context who did express more affective goals 
appeared to be caught in a conflict between their goals and the context in which they operated. 
Some were still constrained by a belief system which they had internalized from years of tech- 
nological control. There was conflict between their beliefs in literature, and their beliefs about 
how to keep track of literacy development. 

Not only did teachers' methods of keeping track of children's development tend to re- 
flect their goals and values, but both appeared related to their evaluations of their own effec- 
tiveness. Thus, the fifth major summary statement is: Teachers' evaluations of their own effec- 
tiveness in the teaching of literacy and literature were closely allied to th?ir methods of keeping 
track of children's development. 

We investigated teachers' evaluation of their own literacy instruction by asking "Are you 
doing a good job?" and "What makes you think so?" and "Can you think of a time when you felt 
really satisfied (and dissatisfied) with your work?" The responses to these questions can be 
divided into three main categories of sources of feedback. Inside the classroom indicators con- 
sisted primarily of children's responses to the teacher, to books, to the curriculum, and to each 
other. Those responses took either verbal or non-verbal forms, from casual looks or remarks to 
written responses in journals or on teacher-made and commercial publisher tests. Feedback 
from outside the classroom came from many sources. Teachers self-evaluations were affected 
by comments and requests, and even honors, from various people with whom they associated 
professionally and personally. They also took into consideration ideas gleaned from observation 
of colleagues, attendance at conferences and graduate courses, outside reading, and comparing 
their own performance with that of others. A third category, less often described, was that of 
feedback from a teacher's own intuition. 



Feedback from Inside the Classroom. 

In the more literature-based district, the principal source of teachers' evaluations of their 
own teaching of literacy and literature was, by an overwhelming margin, student's comments 
and behavior. There were varying degrees of specificity in the description of the sources of in- 
class feedback. For example, one teacher simply reported noticing "a sparkle and glimmer" 
from students engaging in literate activities, which she felt had developed over the time they 
spent in her class. However, observation of students as they participated in free choice reading 
activities reportedly gave these teachers at all levels much information about the effects of lit- 
eracy instruction in the classroom. 

The less controlled the environment, the more teachers reported paying attention to 
students' conversations among themselves, about books and related lessons, as indicators of their 
own effectiveness in the classroom. Several teachers noted that the development of collaborative 
literate activity was something they observed and valued. These observations of student behav- 
ior influenced not only teachers' evaluations of instruction, but their views of themselves as 
teachers of literacy. For example, one new second-grade teacher summarized her beliefs about 
her growing assurance as a teacher in this way: 

I ihink that's one thing I feel confident in talking about because I do see the results in 
the classroom, about the way to teach kids how to read. Or what to do when they're 
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having trouble reading. It do feel confident now, because of the results that I can see 
with the kids. 



The emphasis on classroom observation was not limited to the elementary levels. The 
secondary teachers in the less controlled contexts generally reported listening to student opinions 
about their instruction, often directly asking students for their views of instruction. They also 
repeatedly mentioned the development of student's self-concepts in literate activities, the "flow" 
of particular lessons, and the evidence of literacy learning found in writing samples in the class- 
room. Class-wide improvement on classroom tests was sometimes mentioned. 

Some teachers in the highly controlled context cited similar indicators of their effective- 
ness. For example, a high school teacher reported that an indicator of successful instruction was 
that the students "begin to help each other". However, the emphasis on students' competitive 
attainments and classroom testing was substantially greater the more controlled the context, and 
more frequently linked directly to teachers' remarks about observations of student behavior. 
For example, when asked how she knew that an instructional change she had made in her class- 
room was working, one teacher answered: 

Oh, because students are . . . I'm feeling better about myself as a result of students 
showing . . . increased performance on tests or children sharing information with me 
or being able to give me back that information I've given out to them or that they've 
learned in a hands-on experience. 

Writing contests, spelling bees, and other public performances (e.g. "sometimes my students read 
their book reports on the intercom") were also mentioned as indicators of teachers' effectiveness. 

Differences were apparent across levels of the literature-based schools. For example, 
there was an increasing emphasis on comparative attainments in shifting from elementary school 
through middle school to high school. Elementary teachers reported attending to indicators 
within the classroom to inform them of the quality of their literature instruction more than did 
middle grade teachers who reported it more than did high school teachers. The teachers in the 
highly controlled context, on the other hand, reported similar indicators of effectiveness at each 
level interviewed. 



Feedback from Outside the Classroom. 

In the literature-based district, positive parental feedback appeared to be valued by 
teachers and it appeared to be taken as the norm. Teachers also felt that lack of negative 
feedback was significant. Examples were given of parental reports of children taking class- 
room discussions of literature home and involving parents. Teachers reported very full attend- 
ance on parent's nights. 

Administrators and colleagues were most often spoken of as supportive, providing ideas 
and encouragement rather than what one middle school teacher termed "critical evaluation". 
Teachers in this district, by and large, appeared to have developed positive images of themselves 
as teachers of literature through working with others as team teachers, in district-wide writing 
workshops, in in-service programs, "cabinet" meetings and middle school academic teams. Even 
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those few teachers who mentioned some frustration - for example, in dealing with bureaucratic 
book selection or purchasing processes (at the high and middle schools) or in going against the 
norm and being heard (in elementary school) - also spoke of the valuable contributions they had 
made to their school's literacy programs and the administrative recognition they had received as 
a result. Similarly, receiving tenure was described as a positive affirmation of teachers' worth 
to the school, and hence the school's acceptance of their literature instruction. Also, monies 
available for conferences and quality in-services (particularly at the elementary level), and other 
encouragement of further education, gave teachers of literature in this district a sense of being 
valued. 

In the literature district, standardized testing was most often looked at as being there to 
provide worried taxpayers with numbers. Teachers sometimes mentioned that they would be 
concerned if their entire class did not perform well on a particular section of the test but there 
did not seem to be a general, heavy emphasis on these tests. Those teachers interviewed around 
test time expressed substantially more concern about the tests and the insecurity they engen- 
dered than did those interviewed earlier in the year. These insecurities are captured in the 
quote in Appendix H. 

Teachers in the highly controlled context more frequently mentioned state tests and 
standards issued by the state. A typical response to our question H Can you think of a time when 
you felt really satisfied or confident about your work?" was that of a teacher of eighth- and 
ninth-grade classes who told us: 

Yes. About two weeks ago the test scores came back from downtown, and just about all 
of my students mastered the skills that they were supposed to have mastered by the end 
of the semester - all except two, out of a class of 53. ..and I was extremely satisfied. 

Parents were infrequently referred to by teachers in the controlled context as sources of 
positive feedback, though one fourth-grade teacher, apparently unusual for this district, did 
mention that she M had a parent who recently said 'What are you doing? My child is starting th- 
read at home'." She ascribed that reaction to her practice of reading to the class frequently. 
This was an exception and mentioned with a tone of surprise. 

In the highly controlled district, administrators and teams of observers (part of the 
mandated teacher evaluation program) were rarely mentioned as sources of feedback for teach- 
ers. The few times they were mentioned were in connection with especially favorable reviews. 
Some teachers spoke of professors and coursework as giving them an indication of how they 
were doing, and several mentioned family members and close friends as 'sounding boards' and 
sources of encouragement. 



Internal Sources of Feedback. 

The teachers in the less controlled context often spoke of a very personal sense of 
knowing that their literacy instruction was succeeding. For example, in the words of one 
elementary teacher: 



Sometimes when I go home at the end of the day, I feel as if things just went very well, 
that the kids were comfortable and everything flowed, and the kids are real interested 
and they're excited, and I feel especially good when I, when ideas keep coming ... 

For this teacher, and some others, this was as important as formal evaluation procedures, if not 
more important, but not as important as the within-classroom sources of feedback. This inter- 
nal feedback was not always positive. 

For the teachers in the more controlled context, intuition and conscience also held an 
important place. They often expressed some satisfaction that they were doing their level best in 
a difficult environment. By way of example, a middle school teacher told us that she knew she 
was doing a good job as a language arts instructor: 

Because of what I feel on the inside. That's what lets me know. Despite the fact that I 
have students who refuse to do their work, ah, the fact that I care, the fact that I con- 
stantly interact with them, try to push them, I feel good about that. 

Another answered: 

Yes, based on my evaluation I think I am. And my conscience- I'm at peace with 
myself and I am at ease with my conscience - and I think that is first and foremost. 

The sixth summary statement is that: Teachers were more or less inclined to admit to 
limitations on their own professional knowledge related to the evaluation of literacy development. 
Most teachers either volunteered or encountered in the interviews, limitations on their profes- 
sional knowledge. The teachers in the least controlled context almost all volunteered limitations 
on aspects of their knowledge of children's development with respect to literature and on their 
knowledge of the teaching of literature. Direct prompts produced many reported limitations. 
For example, there is an implicit limitation when identifying "a student you know least," and 
when asked, "Are any of the students hard to give grades to?" Outside of this direct prompting, 
only two of the teachers in the most controlled setting volunteered limitations on their profes- 
sional teaching knowledge, though all encountered limitations on their knowledge within the 
framework of the interview. For example, although their descriptions of children's literacy 
development were relatively meager, they seemed unaware of the fact. Similarly, although five 
out of six elementary teachers in the most controlled context could not remember any titles of 
books which the students had read recently, this did not seem problematic to them. 

The first type of evaluation qualification referred to a failure to understand a student's 
literacy development. These comments ranged from comments like "There's a lot I feel I need to 
know about Steven" to "I have a gut feeling that Joseph can read better than he does in a basal", 
and more specifically: 

When she comes to a word that she doesn't know, she doesn't seem to have as much 
information about what to do, and I don't feel that I have a real good sense of what 
she's really capable of ...I don't feel that I know myself and I don't feel that piddly 
reading tests, and these little passages that we've had, and the basal are a fair evaluation 
of it. 
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When teachers were asked to describe the literacy development of a student not known 
as well as other students it was common for teachers to ascribe the cause of the problem to the 
student being shy or retiring and thus not readily displaying his or her knowledge overtly. Less 
able students who did not like to read or write much also were mentioned as being less well 
known. On the other hand, some of the less able students received the lengthiest descriptions. 
However, some of the teachers in the less controlled contexts also expressed limitations on their 
knowledge of students they knew quite well. One teacher in the literature context expressed a 
qualification that she was coming to grips with, that she had not read all of the books that the 
children were reading, and thus was unable to be sure that they understood them. 

Other evaluation-related qualifications involved concerns about record keeping, relative 
competence with respect to other teachers, lack of understanding of the total assessment pro- 
gram in the school, and the inability to adequately judge progress in literacy learning. Some of 
these teachers in the least controlled district indicated that they were interested in improving 
their ability to follow students' literacy development, an interest that did not arise in the inter- 
views with the teachers in the most controlled context. For example, one first/second-grade 
teacher in the least controlled context mentioned: 

So there are different ways of keeping track of children and I'm always looking for 
something better. And I think one of the teachers does have a better idea, is using a 
notebook with footpages, and I think maybe that would work better... 

On the other hand, some teachers in less controlled contexts expressed an insecurity with 
teacher-developed procedures. For example, one fifth-grade teacher noted that she might want 
to use 'book tests 1 instead. 

A further area in which teachers expressed limitations on their knowledge had to do 
with uncertainty or conflict over the value and consequences of grading practices. For example, 
a fifth-grade teacher in a less controlled context confided that there are indeed students that she 
has a hard time giving a grade to. Her remarks were reflected by many of this group of teach- 
ers. For exrmple: 

Kids that are below grade level, that you think 'do I grade them below grade level, and 
give them an A for being below grade level, or do I give them a 'C or 'D' for being on 
grade level?' I think that's very hard to do, because you want them to build self-confi- 
dence but you don't want to misguide them. Even though they see below grade level 
and they see an A, it's very misleading to parents, and it's very misleading to kids. But 
also seeing a 'D' isn't very rewarding either. 

More than half of the teachers in the highly controlled context also reported that they 
often had difficulty assigning grades which they felt were accurate and useful at the same time. 
For example, a teacher of grades eight and nine told us: 

You don't want to fail them, but you know that they are functionary illiterate. . . But 
you don't want to fail them because this child is doing all that he can do, so you give 
the student a 'D' and deep down you know that that student cannot read and that stu- 
dent may come up with the best answer and you have to reward it. A tough decision to 
make, but I have to live with my conscience and if the student tries, that's all you can 



ask the student, and I'll go with that. 

It seemed that the greater the acceptance of technological control, or a technological 
perspective, the less difficulty teachers had in giving grades. For the most technologically ori- 
ented teachers, grades came down to a brief period of time with a calculator. 

Tb<» difference between the teachers in the more and less controlled contexts in their 
willingness to express insecurities or qualifications on their knowledge also extended to areas 
beyond the area of evaluation, such as their teaching practice. A fourth-grade teacher in a low 
control context prefaced her interview with a remark that suggests two possible reasons for this 
difference. She noted that: 

This is the first year that I have been totally involved in a 3elf-selection literature-based 
program with the children. Our district is moving very much towards that, and hoping 
that everyone will eventually buy into it and move toward it, and a really nice tiling 
about our district is that they don't force you into things, they usually let you ease into 
it, with your own level of comfort. So it just might be helpful for me to say to you, 
that what I'm gonna be describing is my first time through doing it this way, so to 
speak. And, I'm very much feeling my way with it... 

In other words, many teachers in the least-controlled situation expressed an awareness of 
being involved in voluntary and positive, but nonetheless somewhat unsettling, changes in their 
instruction. Furthermore, the supportive nature of the district may well allow teachers to re- 
flect on their practice with less threat than in other districts, and thus to look more closely at 
the parts of their practice with which they are less comfortable. 



Summary and Concluding Comments 

In this study, we found that teachers by and large express certain goals for their teach- 
ing of literature, and reflections of these goals are what they look for in their evaluations of 
their students' development, and of their own effectiveness. Thus, teachers who valued a love 
of literature tended to love literature themselves, and looked for that love in their students first. 
When they saw it they felt successful. Those who valued particular skills or knowledge looked 
first for those. This is a reminder of the value-laden nature of evaluation. However, we do 
not know how these goals are conditioned by the context in which the teachers work. They 
may not have begun with these goals, bur may have developed them, in part, by working in a 
situation that fostered them. 

Teachers' knowledge of literature was clearly related to their evaluations, and framed the 
way they viewed children's development. However, this relationship was not without constraint. 
The context in which teachers worked influenced whether or not their love of, or knowledge of, 
literature influenced the way they evaluated children's development. In the upper grades, 
teachers were responsible for keeping track of the development of perhaps 150 students. Even 
though most of these teachers knew literature well, their descriptions of students' development 
did not reflect this knowledge. One high school teacher expressed a qualification on her profes- 
sional knowledge along with a critical caveat which speaks to this problem. She said: 
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Like with 45 minutes once a day I don't know how to do it [teach literature]. I don't 
know that my students at the end of the year know how to think any better than they 
did when they walked in. I guess maybe that's the problem with teaching English is that 
I ... progress is very slow and very gradual, and I'm easily frustrated ... So the rewards 
are just not visible. 

A similar contextual issue was apparent in the highly controlled elementary schools. 
Though teachers were often unaware of what books children were reading and were unable to 
recall books which had been read, the school day was so constrained by the required use of the 
basal reader with its tests and workbooks, as well as by other curricular considerations and the 
various standardized tests that were mandated, that it was difficult for teachers to find the time 
to read books or allow the students to choose books to read. When asked "Have any books been 
read lately in class?" one teacher responded: 

No. None. I desire to do that but there is absolutely no time with the pacing. It makes 
it impossible. There is no time to ... my concern has always been ... if they hired me 
to teach. . . I wish that they would just let me teach. I'm not saying that there shouldn't 
be any guidelines, but as far as what you teach when, I wish we had more flexibility. 
As long as you cover the bulk of the material over the course of the year, and I have 
things I would love to expose my students to, but time doesn't permit. 

In addition, literature was not supplied to the classrooms in this district and money was 
often mentioned as a problem in obtaining copies of books. Apple (1986) describes this situa- 
tion as "intensification" in which "there is so much to do that simply accomplishing what is 
specified requires nearly all of one's efforts" (p. 44). One of the consequences is that there is 
limited time to keep up with the knowledge in one's field, in this case, children's literature. 

When teachers were constrained to follow a basal reader and evaluation was test-driven, 
even though they might know and love literature, this knowledge did not show up in their 
descriptions of children's literacy development. Where literature pervaded the classroom, teach- 
ers viewed students' development through the books children were choosing to read and discuss, 
nd through the reflections of those books in their chosen writing. In literature-based class- 
rooms individualized pace of progress through literature was encouraged, noticed, and recorded 
by teachers. Where tests were stressed by the district, teachers' descriptions of development 
were more likely to emphasize tests, competitive attainments and test-like language than were 
those of teachers with a greater emphasis on literature who tended to emphasize enthusiasm, 
cooperation and choice of literature. 

In addition, their descriptions of the students' development were relatively scant, reflect- 
ing a lack of detailed knowledge of the students and a less personally involved relationship. 
Teachers who had incomplete freedom from technical control provided descriptions of more 
able students that reflected that freedom and the knowledge of literature, whereas their descrip- 
tions of the less able students did not. It may not be unrelated that suburban teachers were 
generally allowed varying degrees of freedom from technical and bureaucratic control whereas 
the urban teachers were held in the most extreme control with very limited flexibility. 

Nonetheless, for all teachers, the primary source of teachers' kn< ./ledge about students' 
developing literacy was their observations of student behavior and student talk, though in a 
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litera" .-based classroom these observations were different from those in a classroom using a 
basa: .eader. Teachers in the more controlled environment turned to tests for feedback about 
students and themselves before the teachers in the less controlled settings, who tended to turn to 
feedback from parents, colleagues and administrators, much of which tended to be supportive. 

In our data there appears to be, then, a relationship between teachers' knowledge and 
valuing of literature and their descriptions of children's literacy development. This relationship 
was moderated by the context in which the teacher worked. Teachers' evaluations of their own 
effectiveness in teaching literature are similarly colored by context. Both the source of the data 
for the self-evaluation, and the nature of the data appear to be influenced. 
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Appendix A 



Examples of the differences in extensiveness of teachers 1 
descriptions of children's literacy development. 

A first grade teacher who ran a literature-based classroom described a child's literacy 
development as follows: 

Kurt has quite a speech problem, and his production of words is reflected in his reading. 
When he writes, he has progressed to complete sentences and he has a very definite sense 
of whether he wants to write a personal narrative experience, or if he wants to write a 
fictional story. And when he writes, he has multiple characters often. He doesn't usual- 
ly give them fictional names, but he'll call them what they are. If they're animals, 
they're the dog, the rat, and the fox - something like that. 

There's a lot of dialogue in his stories. And he has a lot of action and a very clear sense 
of setting a beginning of his story, and there's some kind of event (and sometimes 
multiple occurrences and a chain of events), and there's often a real conflict, and 
somebody's going to come out on top or [solve] a problem. And a very definite ending; 
very good closing to what he writes. He is able to create his own story. He doesn't just 
take from the literature, for example, as children are still doing. And he uses common 
expressions in his dialogue. He has a very good sense of sound and letter combinations, 
and things in the words that he actually writes. 

He's very committed to his writing. He really enjoys writing, and he feels very good 
about it. He's very proud of his writing. And, actually, I view his writing as being a 
very good sign of where he'll go with his reading, because his interest level in reading is 
higher than his ability level in reading. In reading he is still struggling in some things, 
say in a basal, and this is where I have trouble. Maybe I'll want to be assessing him and 
Hi get out a primer, and he may be having some difficulty, and yet my gut feeling is 
that he's a much better reader than this, because his writing reflects so much more. So I 
don't understand that yet. . . 

He has a tremendous commitment to reading, and he has a pretty good foundation of 
sounds. He understands even if he's producing the wrong sound, he knows what he's 
reading, and he uses context and he really works at sounds. He understands that vowels 
have different sounds, and so he tries different sounds. And one of the things that I see 
happening is that he continually goes for books that are real impressive looking. I mean 
he brought in Treasure Island. And he does have an older brother, who I think is in 
fourth grade, and he desperately wanted to read James and the Giant Peach , and every 
day that I read some pages, or a chapter or two, he would just jump for that book. And 
the thing is that he knows if he's heard it, if he knows that he's heard what was just 
read, he can't wait to see how many of those words he's gonna be able to figure out. 
So he really enjoys the challenge of reading. And he's just dying to work at it and work 
at it . . . It's a little hard for him to accept that little books can be great fun too and 
very satisfying. 
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A third-grade teacher in a highly controlled context described one student's literacy 
development: 

This little boy, Billy, is a very quiet child. He daydreams a lot and sometimes gets the 
material. I never know how he gets it, but of course, he's been retained, so lots of the 
things we are going over in the classroom he had before. But he's a daydreamer. He 
doesn't call much attention to himself, but when I glance at him, I know that Billy is not 
paying attention. When he's paying attention I know that he's getting it, but his atten- 
tion span is not there to be in the third grade. 



Appendix B 



Examples of teachers 1 comments on children's choices of literature 

He continually goes for books that are real impressive looking. I mean he brought in 
Treasure Island.. .he can't wait to see how many of those words he's gonna be able to 
figure out. So he really enjoys the challenge of reading. 



He said [his] favorite that he ever did was Jhs. Two Bad Ants . That's his favorite book, 
because of the story and the illustrations. He liked the illustrations of that because he 
said they were more like cartoons than, say, Ifafi Polar Express . So he was able to 
compare the types of illustrations that the same author used. 



Bob was just absolutely taken with this one sentence Patricia Riley Giff had used to 
describe Matthew, the character in this story, who's a bed wetter. 

[He] was reading James Howe's book, Howlidav Inn , and I said, 'Joey, have you read 
any of Howe's other books?, and he said, "This is the first, and this is the first book I 
ever read on my own in my life. And I really love it, and I want to read more 
mystery." 

Louis loves Mercer Mayer, and he can't find enough. He reads one book, and he'll take 
this one down, and she gives him another book. 



She was a very capable first-grade reader, and I had said try Beverly Cleary and 
Ramona . She had taken off and read a lot of those, and she had read the whole Patricia 
Riley Giff and Cam Jansen books. And then I thought, "Well what else is there that's 
not too advanced for her?" Because she's still only a second-grader. 



* No description of this type came from teachers in the highly controlled context. 
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Appendix C 



Examples OL th£ differences in teachers 1 framing of ihsk 
descriptions q£ children's literacy development. 

Some teachers framed their descriptions almost entirely in terms of observed behavior: 

Okay, one little boy started out reading seme real simple books, and now I can see that 
in his journal, he's getting more out of even harder books. Or talking to me more about 
what he got out of the book, and doing a lot more comparing of books, because he's just 
reading and reading and reading, but he's able to tell me more than he used to be. He's 
more specific. I see a lot of the reading in his journal. He wrote a story about [how] 
once upon a time there lived a little old woman - that type of thing, which he doesn't 
just get without hearing it someplace else. He's using some of that; the story lines to 
write his own, do his own writing. So I do see it in his writing. It's continued on, it's 
more specific, it's not as vague. 

I can definitely see change, he's a very hard worker. He's very picky about what he 
does, and it's hard to say why ... but I can see definite growth in his type of writing and 
type of reading. 



A fourth-grade teacher in a literature classroom began a lengthy description with an 
assertion of a role: 

When I fairly quickly said Linda, it's probably because ... of all the kids I've come 
across in a long time, she is a reader. She loves to read. She just devours books, she 
knows authors, she remembers books that she read, you know, two, three, four years 
ago, with a lot of detail. She can talk about characters, she's very good at summarizing. 



Some teachers' descriptions, such as this first- and second-grade teacher's, began with a 
historical or personal contextualization: 

Well last year he was in my first grade. He had been placed. He was in a first grade 
in another school district, and after two months the teacher said, This child should not 
be here." So they wanted to put him back in kindergarten. And his mother wanted him 
in school all day, I think for babysitting purposes. She didn't want him to go half-day. 
So they put him in kindergarten half a day, and the other half a day they put him in 
some type of a writing program. 



Similarly, a first grade-teacher in a highly controlled context began with: 

Paula is five years old. She comes from a troubled home. Her mother walked out when 
she was two and she was slow in developing. She lives with her grandmother and her 
father. 
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Some teachers descriptions were framed by traits. For example, one third-grade teacher 
in a controlled setting began each description with, and based them on, traits: 

Lisa is a very verbal child. Her confidence about whatever we're doing ... I could tell 
whether she understands or not because she's more apt to speak out. And when she's 
quiet, I know that Lisa is not paying attention, or she doesn't completely understand 
whatever it is we are doing. And that's in all her subjects. She is a very bright student. 
I know what she knows, because when she knows, everybody knows she knows. 



A first-grade teacher in a less controlled context, who used a b^al reader, framed her 
descriptions with "levels": 

Steve entered as basically a non-reader, so he started at the first pre-primer level. And 
it's as if he suddenly discovered ... that this was his meat and potatoes, because he just 
went with it. I mean he's reading at about a 3.1 level now. 

Similarly, an urban eighth- and ninth-grade teacher began her description with: 

Tina T. functions at about the 7.5 grade level and she is very excited about her class. 
She makes good grades. She makes As and Bs. 

Other teachers framed their descriptions in terms of the literature the students were 
reading: 

So he would kind of stick with ... Three The Business Letter , those kinds of 

books. He pretty much stuck with those. Then gradually he branched out a little bit ... 
Fortunately is one of Mercer Mayor's books, and the C*:f r ord books, and that type of 
thing. 
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Appendix D 



Examples a£ descriptions q£ children's literacy 
development made fc^ teachers knowing more and leas, about literature. 

One teacher, when asked, could not think of three authors with whom she was familiar. 
She had no favorite authors. When asked to give examples of things she thought students should 
have learned about literature when they left her class, her response was: 

Now we don't emphasize too much literature. Now when it comes to the different 
types of books, the children like the simple books, those that can't read well ... So I just 
don't know how to answer that question. 

Her description of a student she knew well contained no mention of literature 



Another teacher in a highly controlled context noted that she had no favorite authors 
herself and had little knowledge of children's books. When asked for examples of things her 
students should have learned about literature when they leave her class, her response was: 

The mood ... the mood of the author. I haven't really thought about that. 

When asked how she knows how her students are doing in reading, she commented: 

From testing. From printed testing. And from question and answer. And in two or 
three weeks we can test again to see if they have retained information or skills which we 
have taught. 

Her descriptions of children's literacy development similarly contained no references to litera- 
ture. When asked to describe the literacy development of a student whom she knew well, she 

said: 

From the beginning of her coming into my room, she could not read. She's twelve years 
old - third grade - and very little sight vocabulary. Now she's able to read - not flu- 
ently, but she can read. And she's real proud of herself. She's made progress. She'll 
still be placed in special ed in a couple of weeks, but I do see progress. 

Teachers who knew a lot about literature tended to show this in their descriptions of 
children's development in different ways. For example, the following teacher's description of a 
student shows her use of literature as reference points for his development: 

So he would kind of stick with ... Three Bears . The Business Letter , those kinds of 
books. He pretty much stuck with those. Then gradually he branched out a little bit ... 
Fortunately is one of Mercer Mayor's books, and the Clifford books, and that type of 
thing. 
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Now, when he came in in September, as a second-grader, he kind of was kind of again 
sticking with the things that he kind of felt secure with, like the Frog and Toad Togeth- 
er... And now, I guess he'd have to be considered one of the Patricia Riley Giff experts. 
Because that . . . those books are not real easy, you know, for him. But he just likes the 
character so much. I think he's in love with Ms. Rooney, with her classroom that she 
has. He's willing to spend that extra time getting to know more about those children in 
that room, in that book, and he really tries very hard to be very self-sufficient reading 
those books of hers. . . He's tried the mystery ones that she's come out with. 

I think he needs a lot of realism in his life, because he comes from, you know, a situa- 
tion where he's kind of had to fend for himself from time to time. I don't know wheth- 
er he kind of identifies with some of the kids in Ms. Rooney's room and the problems 
that they have, because she makes them so real and modern ... I think that's why that's 
his favorite person right now. 

But he's also willing to try a lot of other ones ... When he comes to something in that 
Patricia Riley Giff series that he doesn't know, he realizes that he's got to rely on a lot 
of the strategies that he's built up. There are not many pictures to give, so its not like 
he could rely on the old handy thing which he was relying on. 



Appendix £ 



Examples of the differences between teachers' 
descri ptions of !mme_r_ lgvgJ! and "lower level" students. 

An eighth-grade teacher in a literature-oriented context who knew a great deal about 
literature, described two students he knew well, one an "upper level" student, the other a "lower 
level" one as follows: 

I have another upper level student who's moving from Christopher Pike. Finally ... well 
she's trying to read a Danielle Steele book because somebody else is reading a Danielle 
Steele book, and she didn't really like it. She said she liked some of the history in it, 
buy she didn't like the, she called it "sap." So I said alright. But she read Hje. Red 
Badgfi Pi Courage ... and she wanted to talk to me about [it]. And she didn't want to 
talk about 'this is what the story is about.' Usually when we talk about a Christopher 
Pike book she's saying 'this is what it was about.' This time she wanted to talk about 
why he chose to write the story. Why he chose to write the story to begin with. And 
what she thought about that, and what she thought about war, and she brought it back 
to a grandfather that was in a war ... so she was really, she has lots of different ques- 
tions, and I think that different piece or even that different kind of reading - whatever 
it was - caused her to ask more questions. I see her literacy moving, ju-t because she's 
asking different kinds of questions. She's seeing different kinds of things. Going a 
little bit deeper maybe than what the plot was. 

I'll talk about someone who's a lower level student ... He happens to be a remedial 
reader. He's progressing. I can tell that already. He doesn't see reading so much as a 
chore any more, and he doesn't see going to remedial reading as something that the 
dumb kids do. And he reads out loud better. He can read out loud, he can pronounce 
words better. He knows what some of those words mean. So he can not only pronounce 
them, but he knows what they mean also. And he's more willing to volunteer to answer 
a question because he feels more confidence that he's understood what he's read. And I 
think those are - not that he didn't have any of those things when he came in, but they 
were weaker, and I can see that he's progressing in those areas. 
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Appendix F 



Examples showing the relationship festwfisa mala and values and 

wavs of keeping track of teaching and learning. 

In this Appendix, an extended (but still truncated) example of a single teacher is given 
to show the consistency between goals, values and ways of keeping track of teaching and learn- 
ing. A second, briefer example shows a contrast. 

A first-grade teacher in a less controlled context: 

Interviewer: 

Can you give me some examples of things ycu think students should have learned about 
literature by the time they leave your classroom? 

Teacher: 

I think they should enjoy it, number one, and see themselves as being able to enjoy it as 
readers. And have a lot of different authors in mind, and a lot of different types of 
books in mind, so that they can go to a library and not be lost, and say 'Well, where do 
I go?* or, 'What type of book do I look for?' 'I can't find a good book,' - you know, 
that type of thing. I definitely wouldn't want them to say that. I would want them to 
know where to go and what to look for, and have just a lot of information as far as 
authors, and different types of stories, and different range of books as far as being easy, 
just right, and hard, that type of thing. And a lot of exposure to all different types of 
literature, both from me reading to them, and books that they choose themselves. 

Interviewer: 

How do you know how the students in your class are doing? 
Teacher: 

Well I evaluate them in a lot of different ways. I conference with them. Like I said, I 
try to do it once a week. I keep a notebook, and divide it into [gets notebook] ... I keep 
this in a binder. And it's different than the way I did it last year. . . [critiques in detail 
last year's method] and it was just too cumbersome, so I went to this plan, where I have 
the kids' names on, so it's easy to find. I can just flip to a child, and I put the date, the 
title of the book they're reading. I do a running record: I usually put some comments 
about how they read, the pace, whether they were comfortable with the book, where 
they got the book ... sometimes I put that, I don't always. Lots of different things - what 
type of strategies they use, did they use context, did they use decoding, and then I also 
write down where they have problems. Like if there was a word they missed, I write 
down that word. If I notice that they're missing the same word with this particular skill, 
then I write down this type of skill, like, a certain beginning sound, consonant sound, or 
digraph or something like that; I write that down. So then I keep this, and I can com- 
pare how they were doing. Sometimes they choose the same book. If they were work- 
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ing on the same book, which maybe a couple of weeks before was too difficult for them 
and now they're fluent in reading it, then I've known. 



I can usually tell how a lot of it is in my head, which sounds - I know everything 
should be on paper. But I don't dare say that to a lot of people, but it's true. A lot of 
it is, I mean I try to put as much here as possible, but after a while you just get to 
know them from hearing them and, I try to listen to them read aloud. I don't always do 
a running record ... A lot of times in the morning when they come in I'll say, "Who 
wants to read a book to me, H and they'll go, H I do," "I do," H I do." So then they just sit 
down, like on my lap or something here, and they're just reading to me, and I'm not 
taking notes, because I like them to read to me. 

So a lot of times, I'll just say - like if I have five minutes before, and I don't even want 
to bother with this - I'll say, "I've got one or two minutes, can you read part of that 
book to me? H And I'll just listen to them read, and I make a mental note of how they're 
doing, and how the book is and what strategies they're using. But it doesn't always get 
in [this notebook]. But this is a good way for me to appraise, and [with] their writing, I 
try to do the same type of thing; make notes here about that, but ... they have writing 
folders of course. But at the end of every month, I have them write a certain piece, it's 
called their writing scrapbook. And at the back of the piece, 1 write down what I no- 
ticed they can do as a writer. Like, you know, "As a writer Louis can..." and then I 
write down all the things 1 noticed about [him]: he uses periods, uses some punctuation, 
uses some capital letters to begin sentences, can write some words in standard form, like 
can correctly print "the", and "a", some sight words, uses proper consonants, has the 
consonants pretty good, is using some vowel sounds - whatever I notice about them. 
And then I have them do that at the end of every month, so then I look back and I 
compare what they wrote in September, to what they wrote at the end of November. 

I think it's easier for me to lay out more, to see what they're doing, than just the writ- 
ing folders. Because in their writing folders, they keep everything. They just keep 
stuffing things in them, and they're like this, some of the kids. I plow through them, 
and here's what they did one day, and here's what they did back in October. And it's 
harder for me to really see, so this way I found that it's easier to really look at the 
writing, at the end of every month, and think what they did that month, and how they 
progressed or what they're doing. The same thing with their journals, which are ongo- 
ing, which I look at and see those types of things. 

Interviewer: 

Do you give these students grades in language arts? 
Teacher: 

Uh huh. 
Interviewer: 

How do you go about that? 
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Teacher: 

Well I base it on my observations, how their running records are, what we see. We 
have to give them a grade level based on, you know, how they do on a certain passage, 
and how they do on the questions and that's usually the instructional level we have to 
give them. But how I base their grade - a lot is based on their effort, and if they're 
using their strategies for reading, their interests in it, if they're really trying. I seldom 
give a child a four, unless he's really just wasting time and goofing off and doing noth- 
ing, or if he's really capable of doing a lot more and he's not using it. But even a three 
is something which I don't like to give unless I think that ...they really need to improve. 
You know, to show a growth. The report card is really hard to do. It's just, you know, 
always seems ... a kid has to be perfect to be getting a one. And usually a two - that 
seems awful close to one. 

And when I sit down and do them, it takes me forever ... My husband thinks I'm crazy. 
He says, "I can't believe you go nuts over deciding whether to give them a one or a 
two, H but if its important to them, you don't want to make a mistake. I try to look at 
that kid and the way they are most of the time, and make a decision. A lot of the kids 
who could do better get a three. Three isn't bad. Sometimes they're doing these things, 
but it's something they need to work on, being more of a regular that they do. 

Interviewer: 

Are there any students who are more difficult to give grades to than others? 
Teacher: 

Well, some kids just naturally know more. They talk more, for instance. They come 
up to you and share more things with you. They're always anxious to hear you, they 
want to read to you all the time. And they're just the outgoing type of person. You just 
get to know that kid a lot quicker than a quiet child or a child who doesn't show emo- 
tion, show enthusiasm. It's harder to get, to really get, a feel for that kid. I mean even 
though I do the same types of things with them, it's sometimes harder to pull them out, 
and to really get to feel like you really know them well. 

Interviewer: 

Can you think of times when you've felt satisfied about your teaching, or confident? 
Teacher: 

When the kids do a particular thing. Like whjn you see a child, just you can see their 
reading improve ... A book you know they couldn't have handled two or three weeks 
before, and a couple of weeks later they're reading that book, and doing a real good 
job. That type of thing. Or when they write you notes, they say H I like it when we do 
this and that or certain things. 1 * That type of thing. When they produce something. 
Like one little girl goes home and on her own, on their own they come in with a 



project. Today they came in and worked on the science project this weekend. That 
makes me feel good, because I didn't tell them to do it. But they want to learn and 
they're doing it completely on their own. And they want to share it, and I give them 
time to share things like that. 

Those types of things, or when they see themselves as being all the same, as a communi- 
ty of learners ... Last year we were pen pals with my son's class in another school, so we 
used to write back and forth all the time with his class. And when his class would write 
to my kids, they were in a typical first-grade reading group situation, and one little boy 
wrote to one of my kids in here, and said, "I'm in the top group," you know, "What 
group are you in?" And the little boy came up to me and said, "I don't understand this. 
What does he mean?" He didn't even know what it was because we weren't doing 
that ... And I said to him, well how do you think you'd answer this? What would your 
response be? And he said, "Well I think that we're all the same, we're just one big 
group." And that made me feel good. 



Appendix G 



Examples from teachers 1 interviews showing the influence 
of external testing in a highly controlled district, 

My first class is language skills. [I have] 25 students and they range from reading levels 
between a 4.0 and a 7.5, and they are a very interesting group to work with - they 
demand a lot of attention because their attention span is so short. After first hour - 
that's the language skills class and that's a small class and we can work effectively - the 
second hour I have an eighth-grade class and that's language arts. That class is prepar- 
ing to take the eighth-grade state test, the BRST and the Iowa Test of Basic Skills. On 
the ninth-grade level, they are prepared to take the TSP, the Test of Scholastic Progress. 
At the fourth hour I have the language arts again, and that's the eighth grade, and they 
have to prepare for these skills. That fifth hour is the language skills, ninth grade, and 
the sixth hour I have to teach them the skills that they need, follow the objectives, and 
prepare them to take the TBS, the Test of Basic Skills, which is the exit exam for [this] 
school system. They're expected to be prepared in the class - the TBS prep class - so 
we have to constantly have these objectives in mind as we teach on a day-to-day basis. 
On the eighth-grade level we have a set of objectives that we have to teach in order for 
them to pass the BRST and the ITBS ... and also the reading and the literature. The 
eighth-grade level we use the basal reader Encore , so we have the literature and the 
skills. 



An urban sixth-grade teacher commented that the problem with her students not reading 
enough literature was: 

I think that they are going to be at a disadvantage... because there will be ... a test that 
they might take sometime that will make reference to those things and they won't have 
had any exposure to it .., So they will totally be at a loss. 

An urban first-grade teacher indicated that she used weekly tests more than direct 
observations to inform her planning for instruction: 

I give a test every Friday in every subject ... and that would be for me so I can get an 
idea of where to go the next week as far as lesson plans. And the end of the month 
tests, those are more of a composite. That's the one I would count in my grade books, 
but the weekly ones, they let me know what I have to go back to rev<?w, or if I can go 
on with instruction. 
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Appendix H 



An example of ihS. kind q£ insecurity noted th? 
tgaghfilS ill tlM l e P s controlled context with respect to testing. 

A first-grade teacher: 

Naturally you think, "Oh, I'm going to be judged by how my kids do on the CAT test." 
Although they don't really talk about it much, it's still there, and a couple of times Mary 
had said to me, "Oh I hear your low kids are gonna pull your whole class down on the 
CAT tests, and they're not going to look good." And I say "Oh no." And I kept think- 
ing, "But I got those five good one's, and they're gonna pull the class up, so hopefully it 
will average out somewhere in the middle." I kept thinking that all year, which is about 
what happened. The five really top ones did terrific on the CAT tests, and the ones that 
I didn't expect to do real well didn't do very well on the CAT tests, so they did pull 
them [down]. So the whole class came out pretty average, but yeah, I was real con- 
cerned about it. And never having seen them before - that's a real disadvantage, 'cause 
I mean Ruth and I really felt like we were at a loss because some of the other first- 
grade teachers had known what was on the CAT tests, and they would review a particu- 
lar thing, because they knew it would be on the CAT test, and we didn't. 

I spent a lot of time before the CAT test, like maybe the month or so before teaching 
them how to take a test ... I was afraid that they were not going to know how to do this 
type of test. You know, read a passage and answer questions. I had never asked them 
to do that. As far as their personal reading was ... they were responding, they were 
doing a lot of writing and talking about what they read. But I didn't ask them a defi- 
nite question where there was only one answer. That was never part of our program ... 
The first couple I did with them. "Sometimes we have to do this type of reading. Some- 
times you're required to take a test." And they said, "Oh we know you wouldn't give us 
that test if you didn't absolutely have to." We feel terrible. I really wouldn't except that 
I have to because its part of my job, and I said [the principal] would be very upset with 
me if I didn't give you the CAT test ... They ask you a question, there's only one an- 
swer. 

I thought they were gonna blow it. "They're gonna do terrible. My class is gonna do 
awful, that's it. I could lose my job." 



Appendix I 

Teacher Interview Questions 

Describe a typical day in your language arts program. 
Do you group children for reading or writing? 

On what basis? 
What different kinds of things are read in your classroom? 

What books have been read lately in class? 

Can you think of three of them in particular? 

Is one of these different from the other two in some way? 
Is there any other way to classify those three books? 
Do children in your class have favorite authors? 

Can you give me an example of a particular child? 
Do you have a favorite children's author? 

Can you give me some examples of things you think students should know about literature when 
they leave your class? 

Is this class different from your last class? 

How do you know how students are doing in your class in reading and writing? 

Do you use classroom tests? 
Which student do you know the best? 

Could you describe his/her literacy development? 

Could you describe a child who is quite different from that? 
Which student do you know least? 

Why do you know this student least? 

Would you describe his/her literacy development? 
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Are there any students who you feel are not progressing well? 

Why do you think so? 
Do you give students grade: for their reading/ writing/language arts? 

How do you go about this? 

Are any of the students hard to give grades to? 

Can you give me an example that illustrates this? 
With what information are parents routinely supplied? 
What information do parents most frequently request? 

Could you describe some times when you really feel satisfied or confident about your work? or 
accomplished? 

Could you describe times when you really feel dissatisfied or have wanted to give up teaching? 
Have you deliberately changed some aspects of your teaching? 

Could you describe one of these? 

What prompted the change? 

Did it work? 

How do you know? 

How did you feel about making the change when you first started? 
Do people or things outside your classroom influence your teaching? 
Has your view of yourself as a teacher changed over time? 

How? 

What prompted the change? 

Do people or things outside your classroom influence your view of yourself as a teac'ier? 
Are you doing a good joo as a teacher of language arts? 
What makes you think so? 
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If two teachers were discussing assessment in your school, what kid of things would I hear them 
saying? 

You use CAT tests in this district. Can you talk to me about the CAT tests? 

Do they influence your teaching? 

Do they influence you as a teacher? 

Do they influence the children or their parents? 

Can you give me some examples? 

Do you take any special steps to make sure the children give their very best showing on the 
test? 

The way you keep track of students 1 progress - is that the way other teachers in the school do 
it? 

As a teacher, what is it that you know that the average person in the street doesn't know? 

Are there any other questions that you think it would be interesting to ask about how teachers 
know their students and their practice? What have I missed? 

Some of these questions were not appropriate for secondary school teachers. These were 
changed or deleted as necessary. 
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